Probability of intrinsic time-arrow from information loss 
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Abstract 

Time-arrow s = db, intrinsic to a concrete physical system, is associated with the direction of information 
© : loss A7 displayed by the random evolution of the given system. When the information loss tends to zero 

the intrinsic time-arrow becomes uncertain. We propose the heuristic relationship 1/[1 + exp(-sAI)] for 
| the probability of the intrinsic time-arrow. The main parts of the present work are trying to confirm this 

Ph . heuristic equation. 



The probability of intrinsic time arrow is defined by Bayesian inference from the observed random process. 
From irreversible thermodynamic systems, the proposed heuristic probabilities follow via the Gallavotti- 



^ ' Cohen relations between time-reversed random processes. In order to explore the underlying microscopic 

mi 

mechanism, a trivial microscopic process is analyzed and an obvious discrepancy is identified. It can be 



resolved by quantum theory. The corresponding trivial quantum process will exactly confirm the proposed 
heuristic time-arrow probability. 
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I. INTRODUCTION 



Both experiment and theory confirm that physical processes are time-reversal invariant in 'sim- 
ple' systems. This invariance may eventually be lost if the system is chaotic, singular, of many 
degrees of freedom, or not isolated [J. It seems plausible now that time-reversal asymmetry (ir- 
reversibility) is always accompanied by some information loss. Yet, little is known quantitatively. 
The present work discusses an elementary informatic mechanism of irreversibility. It leads to a 
simple analytic expression for the asymmetric probability of the two possible directions of time. 

Suppose we use reference-time t to label the order of events but we leave open whether physical- 
time st is passing with increasing or decreasing t, according to the respective time-arrow s = ±. 
We make no apriori (extrinsic) assignment for s. The ambiguity is to be resolved by analyzing 
irreversible physical processes. We consider informatic irreversibility in a sense that the Shannon 
information changes by AI along the process. We call the resulting aposteriori time-arrow intrinsic. 
It belongs to the given irreversible process. It would not exist in 'empty space' at all. In the spirit 
of the second law of thermodynamics, the physical entropy production sAI must be positive, hence 
the intrinsic time-arrow is unique: 



This assignment is only valid if the magnitude |A/| is macroscopic which means that it is much 
bigger than 1 bit. If, however, the irreversibility is weak then we have to be contented with 
a probabilistic intrinsic time-arrow. The main suggestion of our work is that this probabilistic 
time-arrow is a relevant concept and, furthermore, the probability P(s) depends on the Shannon 
information change AI at largely general conditions. I will consider the following relationship: 



If we change the sign of the reference time (t — ► —t) then also the sign of the information loss 
will change (AI — > — AI). Hence the above expression is covariant against time-reversal of the 
reference frame. Asymptotically it yields the unique thermodynamic arrow Q if the information 
loss | AI\ is much greater than 1 bit. On the contrary, the two time-arrows become equally probable 
for a reversible process where AI is much less than 1 bit Q|. The suggested relationship is heuristic 
and lacks a general proof. It is intimately related to the fluctuation theorem P, P] proved for a 
particular class of irreversible processes |5|. On the other hand, it intends to reflect a fundamental 
meaning of the time-arrow in terms of information flow. I am going to prove that the relationship 



s = sign(AJ) . 



(1) 




(2) 
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© follow from elementary statistical considerations provided we assume some further conditions 
to fulfill. 

Sec. [Q] presents the mathematical steps of Bayesian statistical inference adapted to the estima- 
tion of the time-arrow from the observed data. In Sec. lIIII we discuss the inference from irreversible 
thermodynamic process, in Sec. I1VI from microscopic process. The time-arrow is derived from 
quantum irreversibility in Sec. \V\ The Appendix offers a short proof of the fluctuation theorem. 



II. BAYESIAN TIME-ARROW 



Given a statistical system, let X denote a certain random process in a given interval of reference- 
time t. Let X denote the time-reversal of X. Assume that from the principles of statistical 
physics we can calculate the probability V(X) in physical-time! We also introduce the probability 
distribution V(X) of the same random process seen from a reference frame with reversed time: 

V{X) = V(X) . (3) 

The conditional probability distribution of X takes this form: 

f ViX) s = + 

p(x\s) = { : , (4) 

( V(X) s = - 

where s is the apriori time-arrow. Prior to the irreversible process X, the distribution of s is 
symmetric: Po(s) = 1/2. Hence the joint distribution of X and s is the following: 

1 \ P(X) s = + 
P(X,s) = P(X\s)P Q (s) = -{ ^ . (5) 



V(X) 



According to the Bayes rule, the conditional aposteriori distribution of the time-arrow reads: 

1 f V(X) s = + 

P( S \X) = x I , (6) 

r(x)+v(x) jppq s = _ 

which can be cast into the following covariant form: 

= i + e -sD(X) ' ( 7 ) 



where: 
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This Bayesian estimate means that if 1) we know the apriori distribution V{X) of the random 
process X in physical-time but 2) experimentally we observe either X or X with equal probability 
since we have no apriori information regarding the relationship of our reference-time to the physical- 
time then 3) learning X in the reference-time will lead us to the Bayesian probabilistic estimate 
P(s) of the times-arrow. 

Let us calculate the mean fidelity of the estimated time-arrow: from Eqs. ()4l8j) we shall obtain 
the following closed form: 

F ^ P(M X)P ( X M = ( TT ± }m ) p . (9) 

The expectation value should refer to V(X) which is the distribution in the physical frame. We 
can easily derive an ultimate covariant expression of the average Bayesian estimate: 

P(S) = ( l + e -Uo ) ' ^ 

where the average refers already to the observed statistics and the form is valid in time-reversed 
reference frames as well. 

III. THERMODYNAMIC CASE 

Let X be a coarse-grained macroscopic random process in a given statistical system in the 
period [— T, +T] = [ii,t2] and let X be the same process seen from the time-reversed reference 
frame: 

X = {X(t); h<t<t 2 } , 

X = {X(-t); h<t<t 2 } . (11) 

Typically, X can be an irreversible thermodynamic process X(t). Assume that we know the 
irreversible entropy AI(X) produced by the process X. Obviously, the time-reversed process 
'produces' the same entropy with the opposite sign: 

AI(X) = -AI(X) . (12) 

Let us introduce the following conditional distributions: 

p(x\o = r(x)/p 1 (0 , 

P{X\rj) = V(X)/P 2 ( V ) , (13) 
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where P2(v) are the probability distributions of the extreme values £ = X(ti) and 77 = Xfo), 

respectively. In the Appendix the reader finds an elementary proof of the fluctuation theorem 
encoding the asymmetry of the time-reversal X «-> X into covariant equation: 

P{X\rj) = e- M Wp{X\£) . (14) 

Accordingly, the violation of the time-reversal symmetry is exponentially increasing with the mag- 
nitude |AJ| of the irreversible entropy. We are going to show that, via the Bayesian statistics 
of Sec. [nj the relationship 1)14(1 reproduces the heuristic probabilities (J2J) for the thermodynamic 
time- arrow. 

Let us express the r.h.s. of Eq. (JHJ) from Eqs. (|13I14|) : 

D(X) = Al(X)-logj^ . (15) 

For long enough periods, the r.h.s. is dominated by the information loss AI(X), the second 
(boundary) term can be ignored (c.f. :[!]]). In this limit we can write the covariant Bayesian estimate 
l|7|) into this form: 

P (*\ X ) = I + JsAKX) > ( 16 ) 



which on average leads to the covariant distribution 

p M = ( 1 + e -Ux. ) • («) 

Finally, this yields the heuristic form (J2J) provided we can ignore the statistical fluctuations of 
the entropy production around its expectation value AI = (I(X)). This is justified for common 
macroscopically irreversible processes where |A/| S> 1. 



IV. MICROSCOPIC CASE 



Let us consider a statistical ensemble of n S> 1 independent d— state systems characterized by 
the probability distribution p l ,i = 1,2, ... ,d. Let X be an abstract random process as trivial as 
the transition from an initial microscopic ensemble state £ into a final one 77, the time-reversed 
process X will be the opposite transition: 

x = ( v ,0. (is) 
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Let p\ and p\ be the systems' probability distributions within the ensembles £ and 77, respectively. 
Then the change of Shannon information along the process X reads: 

d d 

AI = nl 2 - nh = -nJ2 pi lo g ?\ + n Pi lo S <°i • ( 19 ) 

i=l i=l 

The process X is irreversible if AI 7^ and we should assign the time-arrow s so that sAI be 
positive l[T|). The point is that the two samples £ and 77 may, by chance, not realize the asymmetry 
especially when the shapes of their probability distributions p\ and p\ do not much differ from 
each other. 

Let us characterize the two constituting configurations of X = (£, 77) by the multiplicities n\ 
and n^: 

e=(ni;i = l,2,...,d) , 

J7 = (n t 2 ;i=l,2,...,d) , (20) 
which follow independent multinomial distributions with the respective mean values 

(n[) = np\ , 

(4) = np\ . (21) 
For large n we can approximate the multinomial distributions by Gaussian functions: 



£1 2np| ^ 2np\ J 
We substitute these expressions into Eq. (jHJ) to calculate -D(£, 77), then we calculate the mean value: 

D =-?£^-^)(s-a)- (23) 

Suppose that D(^,rj) is, for very large n, dominated by the mean value D and fluctuations will 
thus be ignored. Hence the average Bayes estimate (fTfljl reads: 

m = ■ («) 

This could become equivalent with our heuristic proposal provided D = AI which is apparently not 
true in general. I was looking for further conditions at least to achieve the asymptotic equivalence 
of D and AI. I concluded to the following elementary assumptions. First, the shapes p\ and p\ 



must be close to each other so that the lowest nontrivial order in Ap % = p\ — p\ will be sufficient. 
Second, the statistics of either £ or rj must be totally random. This sets the apriori time-arrow 
for s = — or s = +, respectively. For concreteness, I consider the case s = + and adopt flat 
distribution for rj 

P\ = \ ■ (25) 

This second assumption is a necessary one, otherwise AI contains a linear term in Ap l while D 
does not. From Eqs. ()19|) and (|23j) the above two assumptions lead to the following results: 

AI = ^J2(A P r, (26) 
z i=i 

and 

d 

D = rid^i^) 2 . (27) 

8=1 

The result is surprising: Z) has come out twice the information loss. 

Mathematically, D is the Kullback divergence between two neighboring ensembles £ and rj and 
it should asymptotically coincide with the information loss between them. The reason of the 
anomalous factor 2 is that we happened to use the Kullback divergence between the composite 
ensembles (£, rj) and (rj, £) instead of £ and rj. This gives a hint how the factor 2 would go away. 
It is interesting to note that the physical resolution has a typical quantum mechanical motivation. 
In microphysics it is conceptually impossible to observe the full quantity X = (£,77). If, e.g., 
the time-arrow is positive (s = +) then 77 is testable and £ is not because its observation would 
significantly perturb the initial preparation. And vice versa, when s = — then rj is testable and £ 
is not. Accordingly, we are going to change the concept of experimental data. In the concrete case, 
we forbid the observation of £. In this sense, we have to redefine the distribution of the observed 
quantities: 

P^,r j )^V(r l ) = J2'P^,r l ) ■ (28) 

Repeating the calculation of the Kullback divergence in the leading order, inserting the flat values 
(|25|) for p l 2 , D turns out to be half of the previous value (J26|). Thus in the given approximation we 
have obtained the identity 

D = AI , (29) 
and confirmed the heuristic relationship ©. 



V. QUANTUM CASE 



Let us consider the statistical ensemble of n S> 1 independent d— state quantum systems where 
each one has the same density matrix p. Let X be an abstract random process as trivial as the 
transition from an initial ensemble £ into a final one rj, the time-reversed process X will be the 
opposite transition: 

X = (fj,i), (30) 

where 

i = Pi ® pi ® • • • ® Pi = /of n , 

if pi and /92 stand for the systems' density matrices within the ensembles £ and 77, respectively. 
The change of von Neumann information during the process X reads: 

AI = nli — nil = — nTr (p 2 logp 2 ) + nTi (pilogpi) . (32) 

The process X is irreversible if AI ^ and we should assign the time-arrow s so that sAI 
be positive (|T|). In order to A/ have a definite sign the two ensembles £ and r) should display 
experimentally significant asymmetry. 

Quantum theory says that if the reference-time is the physical time (s = +) then we can not 
test the ensemble £ but the ensemble fj. And in the opposite case (s = — ) the ensemble £ is testable 
and r) is not. We see that the estimation of the time-arrow s boils down to the statistical decision 
whether the actually observed ensemble is £ = pf n or f) = pf™ whereas both alternatives have 
equal apriori likelihoods. 

We can mechanically follow the Bayes method of the previous chapters. Note, however, the 
typical quantum informatic arguments: this is the way I approached the issue originally. 

The two collective states (|31(l reside in a Hilbert space of dimension d n . According to the 
quantum counterpart of Shannon's code theory [3], in the large n limit such collective states 
become asymptotically equivalent with totally random states restricted for given subspaces. Our 
states (|3*T|) become random states in subspaces E\ and E^: 

i = pf n ~ e" n/l £i , 

r) = pf n ~ e~ nh E 2 , (33) 
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where E\ and E% are Hermitian projectors of dimensions e nl1 and e n/2 , respectively. The di- 
mensions depend on the von Neumann entropies. We are interested in the situations where the 
experimental distinguishability of the above two ensembles would exclusively depend on the dif- 
ference AI = nl2 — nli of the informations. This is obviously not true in general because the 
distinguishability will depend e.g. on the overlap Tr^i-E^)- Nonetheless, the asymptotic forms 
(|3"3*)) suggest simple conditions to achieve our goal. Suppose that one of the ensembles, say 17, is of 
maximum information: 

* = 3 • (34) 

which means that I2 = logd and E2 = l® n - The ensemble 77 is totally random over the whole 
collective Hilbert space of dimension d n . The overlap between E± and E2 becomes trivial. The 
information loss is always positive and the true time-arrow is thus s = +. But we have to find it 
by deciding whether we have tested the ensemble £ or 77 which are of equal apriori likelihoods. 

Now the experimental distinguishability of £ and fj is already trivial. All we have to do is to 
define E\ as observable and to observe it! If the tested ensemble is £ itself then we get 1 with 
certainty since Tr(Ei£) = 1. If the observed ensemble is the fully random 77 then we get 1 with 
probability Tr(£ir)) = e~ A/ and we get with the complementer probability. As we see, the 
complete experimental statistics is determined by the information loss AI. 

Let us turn to the Bayes method of Sec.[n]to estimate the time-arrow s. As we suggested above, 
the observed data is the value E\ = {0, 1} of the quantum observable E\. The probability V(E\) 
stands for its distribution in the reference time with time-arrow s = + and V(Ei) stands for its 
distribution in the reversed time s = —. In the preceding paragraph we established their values: 

T(E 1 ) = E ie ~ AI + (1 - EtXl - e~ AI ) , 

V(E 1 ) = E l . (35) 
Applying the steps of Sec. ITTI mechanically, first we write Eq. JSJ) into this form: 



-D(E!) 



V{E X ) 



V(Ex) ' 

which is then substituted into the expression Q of the mean fidelity, yielding: 



(36) 



F= \ =^ J (37) 

We have used Eqs. to calculate the average. The result implies exactly the form @ for the 
probability of intrinsic time- arrow in function of the information loss. 



VI. CONCLUDING REMARKS 



I proposed a heuristic probability distribution @ for the time-arrow intrinsic to a given irre- 
versible process. The proposed probability is solely a function of the information loss AI. The 
idea itself comes from the the phenomenological fluctuation theorem. Indeed, the concrete form of 
my proposal can easily be confirmed for the intrinsic time-arrow of standard irreversible processes, 
at least in the limit of macroscopic entropy production AI 3> 1. My basic goal, however, was 
the construction of whatever trivial microscopic process which could underly the proposed depen- 
dence on AI. I analyses the irreversible process of the simplest possible structure in classical and 
quantum versions. The quantum version confirmed the proposed probabilistic time-arrow. Let me 
summarize this central result. 

1) Suppose we know that (in physical time) a quantum ensemble £ of n S> 1 identical systems 
of given (also known to us) state transforms into an ensemble f\ of n totally random systems. 2) 
Suppose we do not know at all whether our reference-time is the physical-time or not, and whether 
the 'resulting ensemble' of the above process has been f) or £. 3) We test the 'resulting ensemble' 
and Bayesian inference will give us the time-arrow with fidelity 

F= 1 

l + e-l A/ l ' 

An infite number of conceptual issues could be raised against the presented ideas. I mention and 
discuss only two. First, the assignment of a non-trivial intrinsic time-arrow to a local irreversible 
process is a speculation. Nature might retain the same universal time-arrow for the whole Universe 
independently of the measure or direction of local information flows. Yet, we do not know if 
Nature is that conservative indeed. We learned from Einstein that Nature delegates the issues of 
local geometry to local physical systems. I adopted the hypothesis that this happens with time- 
arrow as well. Second, the proposed confirmation of the time-arrow probability includes Bayesian 
inference. Many would say that inference is subjective. The obtained probability is also subjective. 
Nonetheless, famous arguments using inference have been used earlier to confirm objective statistics 
of quantized fields ^jj. It is, furthermore, a common knowledge that the maximum-likelihood 
inference of the intensive thermodynamic parameters confirms their true equilibrium fluctuations 
in Gibbs ensembles. 

The present work is an attempt to find universal expressions for the hypothetic intrinsic time- 
arrow. There is a hint of the information loss to play the key role. This does not mean that 
we can already claim an experimental significance which should, of course, be inevitable after all. 
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But theory of intrinsic time opens a series of natural questions to study in the future and there is 
apparently a promise of further analytic results. 
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APPENDIX 



Let X(t) denote a thermodynamic variable of equilibrium value X, where A is the relaxation 
rate, and 7 is the Onsager kinetic coefficient. The time-dependent fluctuations of X(t) are governed 
by the phenomenological Langevin equation: 

dX(t) 



dt 



-X(X(t) -X) + V27 w(t) 



with the standard white-noise w(t). The expression 



^L = ^ {X -x)— 

dt 7 dt 



(38) 



(39) 



will be the local rate of irreversible entropy production (information loss) along the process X(t) 
(see, e.g., in Landau-Lifshitz 8], or in 9]). According to Onsager and Machlup IQj, the conditional 
probability distribution of the process X = {X(t); —T < t < T} at fixed initial value £ = X(—T) 
and for equilibrium value X takes this functional Gaussian form: 

<- T tdX(t) 



P(A|e;A)=ex P (-i-/_ 



dt 



+ X(X(t) - X) 



dt 



(40) 



We shall consider driven thermodynamic processes which can be described by the Eqs. (|38I40|) 
with time-dependent equilibrium values {X(t); —T < t < T}. For convenience of forthcoming cal- 
culations let us write down the distribution functional of the driven process: 

1 r T \dx{t) 



P(A|e;A) = exp(- 



47 



T 



dt 



+ \{X(t)-X(t)) 



dt 



(41) 



Obviously the above equations assume physical time t. Let us express the conditional distribution 
of the time-reversed process X starting from X(—T) = 77, driven by the time-reversed function X. 
Namely, we replace X, £, X in Eq. ()41|) by X,r],X, respectively: 



P(A|r ? ;l)=exp(-i-£ 



dX(t) 
dt 



+ X(X(t)-X(t)) 



- 2 

dt) 



(42) 
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Now we change the variable t in the integrand for —t and insert the relations: 



X(t) = X(-t) , 

Xit) = X(-t) , (43) 



leading to: 

~dX(t) 



P(X\r,;X) = ex P (-±l T 



X(X(t)-X(t)) 



dt) . (44) 



dt 

(Recall that this expression would be the conditional distribution of the process had we observed it 
in the time-reversed frame.) The logarithm of the physical distribution (|41|) over the time-reversed 
one (|44|) will result in a remarkable expression: 

log PCTi ? = - [ T (X(t) - X(t))dX{t) . (45) 



P(X\t];X) 7 J-T 

It follows from Eq. (|39j) that the r.h.s. is equal to the total entropy production (information loss) 
of the driven process: 

A rT 

7 J-T ' 

This and the preceding equation yield the fluctuation theorem 

aft 



AI(X; X) = - [ T (X(t) - X(t))dX(t) . (46) 
7 J-T 



P{X\ m 1) = e- AI{x '' X) P{X\i- X) . (47) 
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